Evolving Righteousness in a Corrupt World

Punishment offers a powerful mechanism for the maintenance of cooperation in human and animal societies, but the maintenance of costly punishment itself remains problematic. Game theory has shown that corruption, where punishers can defect without being punished themselves, may sustain cooperation. However, in many human societies and some insect ones, high levels of cooperation coexist with low levels of corruption, and such societies show greater wellbeing than societies with high corruption. Here we show that small payments from cooperators to punishers can destabilize corrupt societies and lead to the spread of punishment without corruption (righteousness). Righteousness can prevail even in the face of persistent power inequalities. The resultant righteous societies are highly stable and have higher wellbeing than corrupt ones. This result may help to explain the persistence of costly punishing behavior, and indicates that corruption is a sub-optimal tool for maintaining cooperation in human societies.Organismic and Evolutionary Biolog

Dynamics of alliance formation and the egalitarian revolution

Arguably the most influential force in human history is the formation of social coalitions and alliances (i.e., long-lasting coalitions) and their impact on individual power. In most great ape species, coalitions occur at individual and group levels and among both kin and non-kin. Nonetheless, ape societies remain essentially hierarchical, and coalitions rarely weaken social inequality. In contrast, human hunter-gatherers show a remarkable tendency to egalitarianism, and human coalitions and alliances occur not only among individuals and groups, but also among groups of groups. Here, we develop a stochastic model describing the emergence of networks of allies resulting from within-group competition for status or mates between individuals utilizing dyadic information. The model shows that alliances often emerge in a phase transition-like fashion if the group size, awareness, aggressiveness, and persuasiveness of individuals are large and the decay rate of individual affinities is small. With cultural inheritance of social networks, a single leveling alliance including all group members can emerge in several generations. Our results suggest that a rapid transition from a hierarchical society of great apes to an egalitarian society of hunter-gatherers (often referred to as "egalitarian revolution") could indeed follow an increase in human cognitive abilities. The establishment of stable group-wide egalitarian alliances creates conditions promoting the origin of cultural norms favoring the group interests over those of individuals.Comment: 37 pages, 15 figure

Inequity aversion improves cooperation in intertemporal social dilemmas

Groups of humans are often able to find ways to cooperate with one another in complex, temporally extended social dilemmas. Models based on behavioral economics are only able to explain this phenomenon for unrealistic stateless matrix games. Recently, multi-agent reinforcement learning has been applied to generalize social dilemma problems to temporally and spatially extended Markov games. However, this has not yet generated an agent that learns to cooperate in social dilemmas as humans do. A key insight is that many, but not all, human individuals have inequity averse social preferences. This promotes a particular resolution of the matrix game social dilemma wherein inequity-averse individuals are personally pro-social and punish defectors. Here we extend this idea to Markov games and show that it promotes cooperation in several types of sequential social dilemma, via a profitable interaction with policy learnability. In particular, we find that inequity aversion improves temporal credit assignment for the important class of intertemporal social dilemmas. These results help explain how large-scale cooperation may emerge and persist.Comment: 15 pages, 8 figure

Developing, Evaluating and Scaling Learning Agents in Multi-Agent Environments

The Game Theory & Multi-Agent team at DeepMind studies several aspects of multi-agent learning ranging from computing approximations to fundamental concepts in game theory to simulating social dilemmas in rich spatial environments and training 3-d humanoids in difficult team coordination tasks. A signature aim of our group is to use the resources and expertise made available to us at DeepMind in deep reinforcement learning to explore multi-agent systems in complex environments and use these benchmarks to advance our understanding. Here, we summarise the recent work of our team and present a taxonomy that we feel highlights many important open challenges in multi-agent research.Comment: Published in AI Communications 202

Biological Simulations and Biologically Inspired Adaptive Systems

Many of the most challenging problems in modern science lie at the interface of several fields. To study these problems, there is a pressing need for trans-disciplinary research incorporating computational and mathematical models. This dissertation presents a selection of new computational and mathematical techniques applied to biological simulations and problem solving: (i) The dynamics of alliance formation in primates are studied using a continuous time individual-based model. It is observed that increasing the cognitive abilities of individuals stabilizes alliances in a phase transition-like manner. Moreover, with strong cultural transmission an egalitarian regime is established in a few generations. (ii) A putative case of hybrid speciation in three species of Heliconius butterflies is studied using a spatial, genetically explicit, individual-based simulation. Given the ecological and selective pressures observed, the hybrid origin of Heliconius heurippa is supported by the model. However, the coexistence of the parental species and the hybrid species is only transient in the simulation. (iii) Optimization and computational techniques were developed during the implementation of a model of adaptive radiation in Anolis lizards. An efficient and accurate numerical integration routine was developed and a parallel implementation was ran on Kraken, Cray’s XT5 supercomputer. These procedures improved the simulation’s running time by several orders of magnitude. (iv) Optimizations, both in execution time and memory usage, are proposed for some genetic operators extensively used in evolutionary algorithms and biological simulations. Speed-up ranging from two-fold to several orders of magnitude is achieved. A statistical analysis was conducted to ensure the reliability of the methods. (v) No-Free-Lunch (NFL) theorems are theoretical results concerning the performance of heuristic optimization algorithms. The characterization of function sets for which the Focused NFL theorem holds is shown. A generalization of NFL results to random algorithms is proven, as well as a new NFL theorem for random algorithms over arbitrary benchmarks